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This study was undertaken to demonstrate the feasibility of applying expert 
system technology to the Navy's H-46 helicopter maintenance process. A 
microcomputer-based prototype known as a computer-aided diagnostic system (CADS) 
was developed for this purpose. Given a helicopter electrical or hydraulic system 
discrepancy, the troubleshooter interacts with CADS to find the cause. The prototype 
CADS was developed utilizing the M.l knowledge-based system development tool by 
Teknowledge, Inc. 

The complexity of helicopter systems diagnosis, and inadequacies of the 
maintenance manuals, often result in unnecessary removal of system components. The 
prototype CADS is intended to demonstrate that a fully developed system, containing 
all the formal and heuristic knowledge of H-46 diagnostic information, could eliminate 
these problems. Also, such a diagnostic system could provide a comprehensive, stable 
diagnostic knowledge base, regardless of personnel turnover. 

This study includes a description of current helicopter maintenance procedures, 
and how the integration of CADS could improve this process. Also included are 
descriptions of expert systems and the M.l knowledge-based system development tool: 
how they work, and their applicability to structured selection problem-solving. The 
development and testing strategies used for CADS are discussed in detail. Results, 
conclusions, and recommendations for further study are provided. 
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I. INTRODUCTION 



A. GENERAL 

Three objectives of the Naval Aviation Maintenance Program's (NAMP) aviation 
material readiness standards established by the Chief of Naval Operations (CNO) are: 

1. Achieve, for all Navy and Marine Corps aircraft, a 78% mission capable rate 
for deployed aircraft and a 73% mission capable rate overall; a 61% full 
systems capable rate for deployed aircraft and a 56% full systems capable rate 
overall. 

2. Reduce the growth rate in Maintenance Man-hours per Flight-Hour 
(MMH FH) to zero and maintain a level no higher than the FY-78 average. 

3. Reduce the growth rate in maintenance and support costs per flight-hour to 
zero and maintain a level no higher than the FY-78 average. [Ref. 1: p. 2-1] 

It is the responsibility of Naval aviation squadrons to meet these objectives with 
optimum use of manpower, material, and funds. Such is the case with the United 
States Navy's and United States Marine Corps' 21 H-46 helicopter squadrons. The 
H-46 Sea Knight is shown in Figure 1.1. 

The H-46 Sea Knight was developed and manufactured by the Boeing Aircraft 
Company's Vertol Division in the 1960s. Since that time many of the helicopter's 
systems have been modified or changed completely. Increasingly complex helicopter 
systems and the diverse nature of Naval aviation operating requirements place a high 
premium on the ability of squadron maintenance people to achieve the aircraft material 
readiness standards. This ability is a function of the training that these people receive 
and the tools at their disposal. Any means that can be used to increase the 
effectiveness of the maintenance process should enhance the ability of achieving these 
objectives. 

Artificial intelligence techniques offer a promising means for helping maintenance 
personnel. Specifically, the type of computer program known as an expert system 
deserves consideration for this purpose. A prototype expert system program known as 
the Computer-Aided Diagnostic System (CADS) has been developed by the author to 
test the value of such a system; that is, to demonstrate feasibility and applicability of 
expert systems technology to Naval aviation maintenance. 



9 



I 




Figure 1.1 H-46 "Sea Knight" Helicopter. 

B. BACKGROUND 

To understand the application of CADS a brief familiarization of the helicopter 
squadrons' current maintenance troubleshooting process is necessary. When a 
helicopter system discrepancy is discovered by pilots, aircrew, or maintenance 
personnel, the discrepancy is documented on a Maintenance Action Form (MAF). 
The MAF is processed by the squadron's Maintenance Control Office, which routes 
the MAF to the appropriate maintenance shop, i.e., hydraulics shop, power plants 
shop, or electrical shop. The maintenance personnel of the shop, who are trained in 
that specific aircraft system, follow the Maintenance Information Manuals' (MIMs) 
troubleshooting procedures to isolate the problem or symptom, find the cause, then 
take the appropriate corrective action. There are approximately 15 separate MIMs 
volumes. Each volume contains troubleshooting procedures for a specific helicopter 
system. 

C. APPLICATION OF CADS 

CADS is designed as a prototype microcomputer-based expert system to aid 
maintenance personnel (or other squadron personnel) diagnose helicopter equipment 
failures. Given a helicopter system discrepancy, CADS will help find the cause of the 
discrepancy. 
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Helicopter aircraft systems are complex. Finding the cause of a helicopter system 
discrepancy can involve difficult and time-consuming troubleshooting techniques. Yet 
maintenance people must be able to repair helicopter problems quickly and efficiently 
to provide aircraft that are fully systems-capable for the next operational commitment. 
When the aircraft problem is complicated, difficult to troubleshoot, and not covered 
adequately in the MI Ms, troubleshooting sometimes becomes the mere changing of 
black boxes and hoping the problem will become fixed. This can result in curing the 
symptom but not the cause, aggravation of the problem, and time and money wasted 
in maintenance man-hours and parts costs. It is anticipated that the use of a fully 
developed CADS could reduce or even eliminate this problem. 

CADS also can be used in the squadron as a training tool for maintenance 
troubleshooting procedures. A fully developed CADS would contain knowledge 
representing the entire maintenance diagnostic expertise of an aircraft community. 
Utilizing the trace feature of this expert system, maintenance persons perhaps could 
observe CADS' reasoning process in finding the cause of the problem, given the 
discrepancy. 

Such a fully developed CADS would also provide a comprehensive, stable 
diagnostic knowledge base, regardless of personnel turnover. The nature of military 
tours of duty creates the problem of loss of corporate knowledge as experienced people 
are transferred elsewhere. The diagnostic skills of maintenance experts would remain in 
the squadron's CADS long after the experts have departed. 

The H-46 helicopter has recently undergone significant systems modifications in 
an effort to extend its serviceable life. Under the Surviveabilitv, Reliability, and 
Maintainability (S,R,&M) Program all H-46 "A" and "D" models will have significant 
systems modifications, and be redesignated CH-46D. A problem associated with the 
S,R,&M Program is the lack of knowledge concerning the modified helicopter systems 
among operational squadron personnel. The CADS developed for this project contains 
the troubleshooting process as described in the updated MIMs for S,R,&M helicopters 
(CH-46D), and knowledge acquired from identified H-46 experts. 

Although the CADS developed for this project is not truly fully developed, it can 
serve as a prototype to demonstrate the feasibility of applying expert system 
technology to the United States Navy's H-46 helicopter maintenance diagnostic 
process. 
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D. OBJECTIVES 



The objectives of this study are as follows: 

1. Develop a prototype CADS to demonstrate the feasibility of applying expert 
system technology to a maintenance diagnostic process. The diagnostic expert 
system program is written by the author utilizing the M.l Knowledge System 
Development Tool (version 2.1) by Teknowledge, Inc. 

2. Demonstrate the CADS applicability to the United States Navy's H-46 
helicopter maintenance process. 

E. RESEARCH QUESTIONS 

The main question addressed in this study can be summarized as: 

Is a computer-aided diagnostic system feasible and applicable to the United 
States Navy's H-46 helicopter maintenance process? 

Secondary questions pertaining to the research areas are: 

1. Squadron applications. 

• How would the maintenance personnel use CADS? 

• How could the CADS knowledge base provide a valuable training tool for 
maintenance personnel? 

• Could the CADS knowledge base contain the corporate knowledge of the 
H-46 helicopter community's maintenance diagnostic expertise, regardless 
of personnel turnover? 

• Can other people in the squadron use CADS program besides maintenance 
personnel? How will they use CADS? 

• Why is development of CADS particularly germane in view of the recent 
H-46 systems modifications (S,R,&M)? 

2. Use of CADS. 

• What interaction is required between CADS and the user? 

• What are the hardware, software, and memory requirements? 

F. SCOPE AND UIMITATIONS 

This research focuses on the development of the CADS prototype program, and 
on its applicability to the helicopter maintenance process. The prototype CADS 
includes knowledge related to diagnoses of the hydraulic and electrical systems of the 
S,R,&.M CH-46D helicopter. The prototype addresses every symptom and problem 
cause for these two systems as specified in the MI Ms troubleshooting tables. It also 
contains a limited amount of the heuristic knowledge of Naval Aeronautical 
Engineering Service Unit (NAESU) technical representatives, and squadron 
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maintenance experts. A fully developed CADS is beyond the time and monetary 
constraints of this research. The prototype CADS software is intended as an aid to 
using the MI Ms. The prototype is by no means a replacement for these technical 
publications. However, ultimately a fully developed CADS could preclude the use of 
written maintenance manuals. 

This research will not include a discussion of CADS implementation issues in the 
helicopter squadrons. Testing the diagnostic expert system program in the squadron 
will be limited; therefore the program cannot be considered completely validated. 

G. ORGANIZATION OF STUDY 

Chapter II contains a description of the Naval Aviation Maintenance Program's 
Maintenance Data System (MDS). Discussion includes maintenance documentation 
procedures, troubleshooting procedures, and how integrating CADS could enhance the 
process, thereby improving readiness of the squadrons. Chapter III discusses 
knowledge-based expert systems, and the M.l knowledge-based system development 
tool. Chapter IV describes in detail the development of the prototype CADS, and how 
it works. Chapter V discusses the results and conclusions of this research. It also 
provides suggestions for further study, and recommendations for the use of CADS and 
M.l. Chapter VI provides an executive summary of this study and its results. 
Appendix A contains a glossary of acronyms used in this study. A sample diagnostic 
consultation session is contained in Appendix B. The prototype CADS knowledge 
base is contained in Appendix C. 
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II. SQUADRON MAINTENANCE: 
CURRENT PROCESS AND CADS INTEGRATION 



Prior to describing how expert systems work and the CADS development 
process, it is necessary to describe the aviation MDS used by the helicopter 
troubleshooters, in order to understand how CADS would be integrated. This chapter 
explains MDS, maintenance action documentation procedures, troubleshooting 
procedures, and integrating CADS into the troubleshooting process. 

A, MAINTENANCE DATA SYSTEM 

The MDS was developed to provide data input to the NAMP for the purpose of 
meeting the aviation material readiness standards established by the CNO. The 
description of MDS is quoted directly from volume II of NAMP: 

The MDS is a Management Information System (MIS) designed to provide 

statistical data for use at all management levels relative to: 

a. Equipment maintainability and reliability. 

b. Equipment configuration, including alteration and Technical Directive (TD) 
status. 

c. Equipment mission capability and utilization. 

d. Material usage. 

e. Material nonavailability. 

f. Maintenance and material processing times. 

g. Weapon system and maintenance material costing. [Ref. 2: p. 11-1] 

The MDS requires that any work done on an aircraft is to be documented by the 
person performing the work. The maintenance person converts a brief narrative 
description of the job into codes and enters the coded information onto standard forms 
or source documents. The source documents are collected and forwarded to a data 
service facility, where the data are converted to machine records. From these records 
Maintenance Data Reports (MDR) are produced, which are periodic report listings 
summarizing the maintenance data. The MDRs are supplied to squadron maintenance 
supervisors to assist them in planning and directing the maintenance effort. 
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Application of the management information provided by the MDR assists in 
identifying, among other things, trend analysis of the following: 

• Areas in which there are skill or training deficiencies. 

• Efficient or inefficient use of available manpower. 

• Parts with high failure rates. 

• Inadequate troubleshooting. 

These four areas are specifically noted as they are most applicable to this research. 

B. MAINTENANCE ACTION DOCUMENTATION PROCEDURES 

One type of MDS source document is the Visual Information Display 
System/ Maintenance Action Form (VIDS/MAF) shown in Figure 2.1. VIDS/MAFs 
are used to document over 1 7 specific types of maintenance actions, as outlined in 
NAMP. Included are such maintenance actions as troubleshooting man-hours, 
removal and replacement of aircraft parts, repair, the performance of scheduled 
inspections, and accumulated man-hours as a result of work stoppage for parts or 
maintenance. 

A VIDS/MAF is originated by the squadron Maintenance Control Office 
personnel or by aircrew, when an aircraft discrepancy is discovered. The form is also 
used to document scheduled periodic maintenance or special aircraft inspections that 
become due. The initiated VIDS/MAF provides for recording, among others, the 
following types of data, quoted directly from NAMP: 

a. Job Control Number (JCN). 

b. Identity of the organization and the work center in which the work is being 
performed. 

c. The type equipment, system, subsystem, and component upon which work is 
being performed. 

d. How the malfunction/discrepancy/failure occurred, and when it was discovered, 
and the action taken to correct it. [Ref. 2: p. 11-3] 

Maintenance Control forwards a copy of the VIDS/MAF to the appropriate 
work center. For example, if the discrepancy is "No.l Generator failure light 
illuminated in flight", the VIDS/MAF copy will be routed to the electrical shop. The 
work center supervisor screens the document, enters applicable data, and assigns 
workers to the task. 

When complex problems occur, troubleshooting often requires a great amount of 
time. In these cases troubleshooting time is separated from repair time by 
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Figure 2.1 VIDS/MAF Showing No.l Generator Failure Light Discrepancy. 
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documenting two VIDS MAFs: one to reflect troubleshooting time, the other to 
record repair time. 

C. TROUBLESHOOTING PROCEDURES 

The maintenance personnel assigned to correct the discrepancy consult the 
appropriate VII Ms volume. In the above example, it would be the Electrical Systems 
volume. Within each of the 15 MI Vis volumes are either troubleshooting tables or 
troubleshooting sections, which outline isolation procedures for system symptoms. 
Maintenance people from the work center perform the isolation procedures, diagnosing 
the discrepancy to find the cause of the problem. Consider the example symptom of a 
No.l Generator failure light that has illuminated on the helicopter's cockpit caution 
light panel. This symptom description would be entered on the VIDS, MAF by the 
aircrewman who discovered it, as shown in Figure 2.1. It is similarly listed in the 
MI Ms troubleshooting tables as "GEN FAIL light on (both generators on) rotors at 
flight RPM" [Ref. 3: p. 3]. 

The maintenance workers search the troubleshooting tables of the Electrical 
System VI I M for this specific symptom, then perform the isolation procedures. A 
portion of the troubleshooting tables, which they would use for this particular problem, 
is contained in Figure 2.2 [Ref. 3: p. 3]. Other than what is contained in the 
troubleshooting tables, the MI. Ms do not fully describe each step of action and its 
consequences. The troubleshooters learn this through training and experience. This 
experience could easily be included in CADS. A flow diagram of this troubleshooting 
procedure for this example is shown in Figure 2.3 to assist the reader. 

According to the MI Ms troubleshooting tables, there are two possible outcomes 
from this isolation procedure, i.e. , 1) the generator comes on line, and the No.l GEN 
FAIL light extinguishes, or 2) the No.l GEN FAIL light remains illuminated. For the 
first outcome, the VII M gives the cause as a supervisory panel malfunction. For the 
second, the troubleshooters perform subsequent isolation checks, to find the possible 
cause of the generator failure. See Figures 2.2 and 2.3. 

This structured, methodical process of searching for the possible cause of an 
aircraft problem is universal in Naval aviation maintenance. Having found the exact 
cause of the problem after performing all applicable isolation procedures, the 
maintenance workers again refer to the MI Ms. Each problem cause has specific 
corrective actions, which are briefly mentioned in the troubleshooting tables. Detailed 
corrective action procedures are contained in other sections of the MI Vis. 
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6. Table 1 includes isolation procedures and 
corrective actions for symptoms identified by 
letters as follows: 

A. GEN FAIL light on (both generators on) 
rotors at flight RPM. 

B. One ESSENTIAL BUS light on (either or 
both generators on) rotors at flight RPM. 

C. One or both ESSENTIAL BUS lights on 
(associated with one generator) rotors at flight 
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D. One DC BUS light on (either or both 
generators on) rotors at flight RPM. 

E. Opposite DC BUS light on (one generator 
on) rotors at flight RPM. 

7. When troubleshooting the AC/DC generation 
systems refer to the following schematics: 

a. AC System - WP 003 00, figure 2. 

b. DC System and Battery - WP 004 00, 



RPM. 


figure 2. 






Table 1. AC System Troubleshooting 


CAUSE 


ISOLATION PROCEDURE 


CORRECTIVE ACTION 


A. Gen Fail Light On (Both GENs On) Rotors At Flight RPM. 


Supervisory 

Panel 

Malfunction 


With generators "OFF" 
cross connect the opposite 
Supervisory Panel into 
malfunctioning system 
using both test cables (figures 
1 and 2) turn "ON" discrepant 
AC system. 


If generator comes on line, 
remove and replace super- 
visory panel. 


Generator 

Malfunction 


With main generators selected 
and both generator switches "ON", 
check for 115 to 120 vac pins A, B, 
and C to ground of the related 
generator test receptacle 
(161 J3) (161J4) or terminals M, 

P, and R respectively of the 
terminal board on generator 
test cable (figure 1). 


If voltages are correct, check 
phasing, contactor control and 
switch control. If all voltages 
are missing or very low, check 
excitation, PMG output and 
feeder fault If one or two 
voltages are zero, check gen- 
erator feeder wires. (NOTE: 
Voltage readings taken at test 
receptacles with generator 
switch in test position are no 
load voltage indications.) 

These readings may not indi- 
cate an RFI generator. 


PMG 

Output 

Incorrect 


With generators off, connect the 
generator test cable (figure 1) 
between the supervisory panel 
receptacle and connector plug 
(241 P5) (241P6) on the malfunc- 
tioning system. At the test termi- 
nal board, check for approximately 
38 vac between ground and term- 
inal E and terminal H. 


If voltage is low, disconnect 
test cable receptacle at helicop- 
ter cable plug and repeat 
measurement. If voltage does 
not rise to approximately 38 
vac, the pm section of the gen- 
erator is defective. If voltage 
is normal, check exciter wind- 
ing. 



Figure 2.2 Electrical Systems MIMs Troubleshooting Table. 
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AC SYSTEM SYMPTOM: 



#1 GEN FAIL Light On 




Figure 2.3 Troubleshooting Flow Diagram. 
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D. INTEGRATING CADS INTO THE TROUBLESHOOTING PROCESS 

Diagnosing helicopter discrepancies is a process which involves more than simply 
following isolation procedures in MIMs. It takes a person with formal knowledge and 
experience to diagnose accurately and to know exactly which parts are defective. Very 
few mechanics in any aircraft squadron are recognized as good troubleshooters. 
Individuals aspiring to be good troubleshooters will constantly refer to the maintenance 
manuals. They will also ask the squadron's recognized experts how effective 
troubleshooting is performed. What happens when the manuals contain incomplete 
information, or have not been updated with current airframes changes? Worse yet, 
when the experts in the unit are transferred, who will maintain their knowledge and 
experience needed for the maintenance effort, and who will impart that knowledge to 
the people aspiring to attain those skills? 

CADS may solve some of these problems, but may result in others as well. How 
would CADS be integrated into the maintenance troubleshooting process? How would 
the maintenance personnel use CADS in this process? 

Several assumptions are necessary before these questions can be answered. These 
include: 

1. Each squadron work center has (or has access to) a Zenith 248 or IBM PC 
compatible microcomputer. 

2. Each work center has a copy of the CADS program contained on 5.25 inch 
floppy diskettes. 

3. All maintenance personnel in the squadron have received training on how to 
operate the microcomputer, and can boot up the CADS program. 

4. All maintenance personnel have received training in the use of the CADS 
program. 

5. There is a qualified knowledge engineer in the squadron to maintain the CADS 
program. 

When the work center receives a VIDS/MAF, the workers assigned can 
immediately boot up CADS to begin troubleshooting. The MIMs will still be required, 
as the prototype CADS does not contain the corrective-action repair procedures. The 
advantage of CADS is that its use could possibly reduce the time required in 
troubleshooting the discrepancy. CADS offers a much faster means of searching for 
the correct isolation procedures for a problem symptom. 

Using the MIMs is often difficult because the maintenance persons constantly 
must turn pages between the troubleshooting tables, the appropriate part diagrams and 
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figures, and the corrective action procedures section. Using CADS on a 
microcomputer with the MI Ms would eliminate the need for constantly referring back 
and forth between sections. The maintenance people can interact with CADS, while 
leaving the Ml Ms open to the appropriate figures and diagrams. Of course, the 
ultimate system would integrate a fully developed CADS, all maintenance procedures, 
and include all figures and diagrams. Through the use of "pull down windows", all 
information would be available to the maintenance person on the computer, literally at 
his her finger tips. This would eliminate the need for manuals. 

Cases of rarely occurring or particularly difficult problems may be beyond the 
expertise of squadron maintenance personnel. The symptoms and isolation procedures 
may not be contained in the Ml Ms troubleshooting tables. The squadron 
Maintenance Officer may request assistance from the NAESU technical 
representatives. The NAESU technical representatives are civilian systems engineers 
with in-depth training and experience with specific aircraft systems. They combine 
formal knowledge with heuristics (rules of thumb) to solve the helicopter's problem. 

The heuristics of these experts are not contained in the MI Ms, but could be 
contained in a fully developed CADS. Historic cases of problems not addressed in the 
MIMs could be acquired from the experts, then programmed into CADS. Should new 
problems occur, they also could be programmed into CADS. This would further 
expand and improve the CADS knowledge base. This aspect of CADS is particularly 
important when there are no technical representatives readily available. 

Availability of this troubleshooting expertise is often crucial to deployed H-46 
units of the Navy and Marine Corps. The Navy has five Helicopter Combat Support 
Squadrons with H-46 helicopters. Four of the five squadrons deploy units known as 
detachments on ships at sea for extended periods (six to nine months). Each 
detachment contains two H-46 helicopters, six pilots, and approximately 20 
maintenance men. It is hoped that the detachment members will possess enough 
knowledge and experience to fix any H-46 helicopter problem that could occur, without 
support of the mother squadron or technical representatives. But what if they have a 
problem that is beyond their expertise? Having a fully developed CADS onboard ship 
would prove to be of immeasureable value to deployed H-46 units. 

There are also situations in which non-maintenance people in the squadron could 
utilize CADS. A common scenario involves an aircraft and crew that fly cross country 
to another military airfield, which has limited or no H-46 maintenance support 
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capabilities. Upon post-flight inspection it is found that hydraulic fluid is covering the 
entire inboard aft section of the helicopter. The exact source of the leak, is not readily 
apparent. None of the crew are aviation hydraulic systems technicians. They call the 
mother squadron (on a Sunday morning) for help. The duty personnel are all 
administrative technicians. Although unfamiliar with maintenance procedures and the 
use of MIMs, the duty persons could provide immediate information to the stranded 
crew. With the CADS diskettes and a microcomputer in the squadron duty office, they 
could boot up CADS, and tell the aircrew the isolation procedures for the symptom. 

The author believes that integrating CADS into the squadron environment could 
improve the effectiveness and efficiency of the squadrons' maintenance efforts. Quite 
possibly the improvement would be reflected in the MDRs. The MDRs generated 
after CADS integration could be compared to those generated prior to CADS. 
Reductions of troubleshooting man-hours, removal and replacement of aircraft parts, 
repair times, and accumulated man-hours as a result of work stoppage for maintenance 
would be possible indications of improvement. 
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III. DESCRIPTION OF EXPERT SYSTEMS AND M.l 



A definition of an expert system is given by Professor Edward Feigenbaum of 
Stanford University, one of the leading researchers in expert systems: 

... an intelligent computer program that uses knowledge and inference 
procedures to solve problems that are difficult enough to require significant 
human expertise for their solution. Knowledge necessary to perform at such a 
level, plus the inference procedures used, can be thought of as a model of the 
expertise of the best practitioners of the field. 

The knowledge of an expert system consists of facts and heuristics. The 
"facts" constitute a body of information that is widely shared, publicly available, 
and generally agreed upon by experts in the field. The "heuristics" are mostly 
private, little-discussed rules of good judgment (rules of plausible reasoning, rules 
of good guessing) that characterize expert-level decision making in the field. The 
performance level of an expert system is primarily a function of the size and the 
quality of a knowledge base it possesses. [Ref. 4: p. 5] 

A. EMPHASIS ON KNOWLEDGE-BASED EXPERT SYSTEMS 

Knowledge-based expert systems can be considered the most popular type of 
applied Artificial Intelligence (AI) systems. Why knowledge-based expert systems? AI 
research has been aimed primarily at creating a machine with the capability to perform 
problem solving by attempting to duplicate the human thinking process. Less abstract 
are knowledge-based expert systems, which focus on domain-specific knowledge used 
for solving narrowly defined problems. 

Fredrick Hayes-Roth, Donald A. Waterman, and Douglas B. Lenat contend that: 

Machines that lack knowledge seem doomed to perform intellectually trivial 
tasks. Those that embody knowledge and apply it skillfully seem capable of 
equaling or surpassing the best performance of human experts. Knowledge 
provides the power to do work; knowledge engineering is the technology that 
promises to make knowledge a valuable industrial commodity. [Ref. 5: p. 3] 

Knowledge engineering can be defined as the technology of building expert 
systems. Knowledge engineers develop expert systems by performing activities of 
problem assessment, knowledge acquisition, and representing the acquired knowledge 
as rules in the system. 
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There are several reasons for emphasis on knowledge-based systems in 
comparison with cognitive reasoning methods, or even conventional programs. First, 
many of the difficult and interesting problems cannot be reduced to an adaptable 
algorithm. Conventional programs process data by means of complex algorithms 
which yield specific quantifyable results. They are written in code only programmers 
understand. Knowledge-based systems must be user friendly by their very nature. 
Highly interactive, they require user-supplied answers to questions generated by the 
program in order to function. Knowledge-based systems allow the user to halt the 
processing at any time and ask why a particular line of questioning is being pursued or 
how a particular conclusion was reached. 

Second, human experts have had a good track record at problem-solving in their 
fields because they are knowledgeable. If a computer can be programmed to have this 
knowledge it follows that it too can have good problem-solving performance. 

Third, knowledge engineers and experts maintain knowledge-based systems, while 
programmers maintain conventional programs. The action of a knowledge-based 
system is to reason through a problem, not merely execute a mathematical model. 
Consequently, the skills of experts in the given field are required in order to transfer 
their abilities to the computer and keep those abilities current. 

Fourth, the knowledge base is readable, and can be modified easily. The experts' 
knowledge can be represented by sets of "if-then" rules written in plain English. 

Barr and Feigenbaum suggest that in order for expert systems to be useful and 
perform at a significant level of expertise they must include: 

• Facts about the domain. 

• Hard and fast rules or procedures. 

• Problem situations and what might be good things to do when you are in them 
(heuristics). 

• Global strategies (methods of approaching any problem within the overall 
domain). 

• Differential diagnoses (methods of breaking specific large problems into smaller 
ones to solve). 

• Possible theories about the domain itself (how and why the domain is the way it 
is). [Ref. 6: p. 81] 

Aircraft troubleshooting readily fits into the context of a knowledge-based expert 
system. Fault diagnosis involves inferring possible causes from a list of observable 
conditions and potential flaws in system components, following an "if-then" format. In 
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the H-46 troubleshooting tables, the observable conditions are the results of the 
isolation procedures listed for each symptom (see Figure 2.2). Each of these 
observables infers a possible problem cause. Given the advantages of knowledge- 
based systems (compared to conventional programming), Barr and Feigenbaum's must- 
have items, plus the "if-then" structure of troubleshooting, a diagnostic system fits 
perfectly into the framework of a knowledge-based expert system. 

B. DESCRIPTION OF M.l. 

According to Joseph S. Yavorsky, the M.l expert system shell operates as 
follows: 

M.l is a sophisticated knowledge engineering tool to aid development of 
knowledge-based expert systems that are diagnostic/prescription oriented. (It is 
capable of containing up to 2500 rules and facts (Ref. 7: p. 1-10]. Typical 
applications contain 100-200 rules]. It is designed to seek a goal defined by the 
programmer to present a single solution or multiple possible solutions to a 
problem. M.l will accept UNKNOWN as an answer, answer questions about its 
reasoning during a consultation, and will calculate certainty factors for 
conclusions. It has a sophisticated user interface with windowing capabilities 
w T hich makes it user friendly, and eases the development of a system. M.l 
requires an IBM PC, XT, AT, or compatible microcomputer, using PC DOS 2.0 
or later with a minimum RAM of 512K bytes and tw r o disk drives. (Ref. 8: p. 24] 

CADS was developed using a series of microcomputers (IBM PC, IBM AT, 
COMPAQ, ZENITH 248), w r ith hard disk drives and monochrome and color monitors. 

Although earlier versions of M.l were implemented in PROLOG, M.l version 2.0 
and later is w’ritten in the C programming language, which is transparent to the user. 
This affords M.l the capability of accessing database and calculating programs using C 
language patches. This capability was not investigated in this research. M.l can 
access other M.l programs. The knowledge base program code written by the 
developer can be prepared on any standard word processor such as Wordstar, and is in 
the form of facts and "if-then" rules similar to written English. 

1. Inference Engine 

Facts are represented as attribute-value pairs called expressions that describe 
the attributes and relationships of objects. The rules represent application of those 
objects in certain situations. Facts and rules characterize formal and heuristic 
knowledge. The M.l inference engine is described by Yavorsky as follows: 
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The inference engine will seek values for expressions by methodically considering 
previously stored conclusions (cached values), relevant knowledge base entries, 
and information supplied by the user. Previously stored conclusions can be those 
facts that never change that are resident in the program, or values that have been 
determined previously during the run of the program. These conclusions are 
stored in what is known as the cache. Relevant knowledge base entries are the 
rules and processes in the program which will determine through inferencing, the 
values for the expression. If values have not been determined by either the 
search through the cache, or by inferencing, then M.l asks the user 

What is the value of: Expression? 

The reference manual for M.l gives a succinct example of the order in which a 
value is sought for an expression [Ref. 7: pp. 4-2 - 4-3]: 

As an example, consider the simplest possible knowledge base, consisting of a 
single knowledge base entry: 

goal = advice 

When you begin a consultation using this knowledge base, the following events 
take place: 

1. The inference engine identifies the goal expression of the consultation and 
begins to seek a value for advice. 

2. M.l checks to see if advice is an arithmetic expression for which it can simply 
compute the value. It is not. 

3. M.l searches the cache for a prior conclusion for advice. As no such 
conclusion yet exists, the search is unsuccessful. 

4. M.l searches through the knowledge base for an entry that can help 
conclude a value for advice. No such entry exists, so again the search fails. 

5. M.l asks a question: 

What is the value of: advice? 
to which you may respond: 

> > sell < Enter > 

6. The system has found a value for its goal expression. M.l displays the 
conclusion, along with its justification, and returns you to the top-level 
interpreter. 

advice = sell (100%) 

because you said so 

M.l > 

The method M.l uses to seek values for expressions via knowledge base entries is 
called backward chaining. [Ref. 8: pp. 24 - 26] 
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Backward chaining is a control strategy that starts with a goal or an expected 
conclusion and works backwards, looking for evidence (other goals or expressions) that 
support or contradict the expectation. Backward chaining is shown in Figure 3.1. 




The M.l reference manual gives a good example of its use of backward 
chaining: 

. . . consider the following simple knowledge base: 

kb- 1: goal = best-color. 
kb-2: if main-component = fish 

then best-color = white. 
kb-3: if day-of-week = friday 

then main-component = fish. 
kb-4: question(day-of-week) = 

'What is the day of the week?'. 
kb-5: if best-color = white 

then wine = chablis. 

When a consultation is run with this knowledge base, the following takes place: 
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1. M.l begins seeking the goal expression, best-color. After first checking the 
cache, the inference engine tries to find a knowledge base entry that might 
conclude a value for best-color. 

2. Finding kb-2, the inference engine then tests the premise of that rule by 
trying to find a value for main-component. 

3. After checking the cache and finding no conclusion mentioning main- 
component, the inference engine locates kb-3 and tries to use it. kb-3 causes 
M.l to seek day-of-week. 

4. The only knowledge base entry that can help find a value for day-of-week is 
kb-4, so you are asked the question: 

What is the day of the week? 

> > 

5. If you answer friday, M.l concludes that day-of-week is equal to friday, and 
notes that fact in the cache. 

6. This causes kb-3 to succeed, and VI. 1 notes that main-component = fish in 
the cache. 

7. This causes kb-2 to succeed, and the inference engine notes that best-color = 
white. Since this is the goal of the consultation, M.l displays its conclusions 
and returns you to the top-level interpreter. 

best-color = white (100%) 
because kb-2 

M.l > 

Had you answered anything other than friday, all the rules would have failed and 
M.l would have indicated that it could not find a value for the goal expression: 

best-color was sought, but 

no value was concluded. 

Note that M.l does not invoke kb-5 even though logically it could use kb-5 to 
infer that wine = chablis after the last conclusion was noted. It does not do so 
because nothing caused it to seek the value of wine. M.l never invokes a rule 
unless its conclusion provides a value for the expression currently being sought. 

An expression is never sought unless it is explicitly declared to be a goal (or 
initial data) or unless it is sought as a result of backchaining from a goal. 
[Ref. 7: pp. 4-11 - 4-13] 

M.l also has a limited forward chaining capability. Forward chaining starts 
with known or available information provided by the user, stored in the cache, or with 
facts in the knowledge base. The goal or conclusions are based on pre-determined 
rules and the available information. Forward chaining is shown in Figure 3.2. This 
control mechanism can also be used to seek expressions and conclusions that are in a 
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sequence-specific pattern. The M.l command whenfound or whencached forces the 
inference process to search rules in a particular sequence. Used in the form 
whenfound(EXPRESSION = VALUE) = LIST 
means that if the EXPRESSION has the specified VALUE, then the LIST, which can 
be one or more other EXPRESSIONS, is true. For example, if the discrepancy on an 
H-46 is "No.l Generator failed in flight'', this is an Alternating Current (AC) electrical 
system problem. A control mechanism is needed to force M.l to exclusively search 
rules that deal with the AC electrical system: 

whenfound(major-system = electrical) = 

{ electrical-sub-system, electrical-sys-symptom ]. 

M.l will interrupt its backward-chaining process to seek values for the two 
EXPRESSIONS in the brackets. In the above example, the user would be prompted 
for values of the two EXPRESSIONS 

Which electrical sub-system is the problem located? 

1. AC system 

2. DC system 

3. APU system 
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The correct response being 

> > AC system < Enter > 

therefore 



electrica!-sub-system = AC system. 

The value for electrical-sys-symptom is found similarly. When it has found those values 
it resumes its backward-chaining search through rules with those specific values. 

2. Uncertainty 

There are cases when the troubleshooters are not absolutely certain that they 
have found the specific cause of a problem. Even having followed the proper 
diagnostic procedures, which led to one cause, there exists the uncertainty that there 
may be another cause of the problem. M.l has the ability to represent and use 
uncertain knowledge. Yavorsky discusses M.l's handling of uncertainty as follows: 



Certainty factors indicate the degree to which a fact is believed as indicated by an 
integer between -100 and + 100, where: 



• + 100 represents complete certainty. 

• 20 represents a minimum threshold of belief. 

• 0 represents no evidence for or against. 

• Negative numbers represent belief that the fact is false. 

• - 100 represents complete certainty that the fact is false. 



Within M.l certainty factors less than 100 (the default value) may arise because: 



• the answer to a question is qualified by a certainty factor, or, 

• a fact in the knowledge base has an attached certainty factor, or, 

• the conclusion of a rule contains a certainty factor. [Ref. 7: p. 4-16] 

As evidence accumulates during a consultation, certainty factors must be 
combined to come up with a single level of confidence for the final conclusion. 
In combining two positive certainty factors, the formula used is: 

CF-noted = CFl + (CF2)% of (100- CF1). 

An example is shown in Figure 3.3 [Ref. 7: p. 4-17]. Certainty Factor 1 (CFl) = 
50 and Certainty Factor 2 (CF2) = 30. So the Certainty of the conclusion (CF- 
noted) = 65 or: 

65 = 50 + (.30) * (100 - 50). 

The combination of two pieces of negative evidence is the same as that for two 
pieces of positive evidence, with the exception that after the calculation, the 
negative is taken of the result. The formula is thus: 
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CF 50 



CF 30 




CF 30 



kb- 1: if main-component = meat 

then best-color = red cf 50. 
kb-2; if preferred-color = red 

then best-color = red cf 30. 
k.b-3: main-component = meat. 
kb-4: preferred-color = red. 
kb-5: goal = best-color. 

best-color = red cf 65 

because kb- 1 and k.b-2. 



Figure 3.3 Combining Two Positive Certainty Factors. 

CF-noted = - (| CF1 | + | CF2 |% of (100 - | CF1 |)) 

= CF1 + CF2% of (100 + CF1). 

For example, for a Certainty Factor 1 (CF1) = -50 and a Certainty Factor 2 
(CF2) = -30 the Certainty Factor concluded (CF-noted) = -65. 

-65 = -50 + (-30) * (100 + (-50)) 

To combine both positive and negative evidence, the two certainty factors are 
added, then the result multiplied by a scaling factor of 100/(100 - A) where A is 
the smailer of the absolute values of the two factors. The formula: 

CF-noted = (CF1 + CF2) * 100/100 - A), 

A = min(| CF1 |,| CF2 |). 

An example is shown in Figure 3.4 [Ref. 7: p. 4-19]. Certainty Factor 1 (CF1) = 
-50 and Certainty Factor 2 (CF2) = 70. A = min(| -50 |, | 70 ]) = 50. So the 
certainty of the conclusion (CF-noted) = 40 or: 

40 = (-50 + 70) * (100/(100 - 50)). [Ref. 8: pp. 27 - 29] 
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CF -50 



CF 70 




kb-1: if main-comoonent = fish 

then best-color = red cf -50. 
kb-2: if sauce = tomato 

then best-color = red cf 70. 
kb-3: main-component = fish. 
kb-4: sauce = tomato. 
kb-5: goal = best-color. 

best-color = red cf 40 

because kb-1 and kb-2. 



Figure 3.4 Combining Positive and Negative Certainty Factors. 

The Teknowledge Reference Manual provides a list of interesting 
consequences, when using the above method for calculating the combination of 
certainty factors: 

• The final certainty factor is independent of the order in which evidence is found. 

• As positive evidence accumulates, the resulting certainty factor approaches, but 
cannot pass, 100. Similarly, accumulating negative evidence can approach, but 
not pass, -100. 

• Once the certainty of a conclusion reaches 100 or -100, it cannot be changed 
again by additional incoming evidence. 

• Certainties below 100 cannot combine to produce 100. Certainties above -100 
cannot combine to produce -100. 

• 0 combined with any certainty factor leaves the certainty factor unchanged. 

• Equal positive and negative evidence will exactly cancel out (except + 100 and 
-100, that cannot be changed once concluded). (Ref. 7: pp. 4-19 - 4-20] 
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This chapter gave Feigenbaum's definition of an expert system. It also 
provided a discussion of why knowledge-based systems have become popular for 
problem-solving situations, the advantages of expert systems over conventional 
programming, and why diagnostic systems meet the criteria for use in a knowledge- 
based system as defined by Barr and Feigenbaum. The remainder of the chapter was a 
description of the basic workings of the knowledge engineering tool M.l, based on 
Yavorsky's discussions. This included the hardware and software requirements for 
running M.l, how the M.l inference engine works, and how M.l handles uncertainty. 
The next chapter will describe the strategy used to develop the prototype CADS 
knowledge-based expert system using M.l. 
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IV. CADS DEVELOPMENT STRATEGY 



The strategy used to develop CADS was derived from a combination of three 
methodologies for building expert systems. They are Harmon and King [Ref. 4: p. 
17S], Hayes-Roth, Waterman, and Lenat [Ref. 5: pp. 159 - 160], and Teknowledge, Inc. 
[Ref. 9: pp. 11-1-11-4]. These methodologies can be summarized in the following six 
steps: 

1. Identify and define a problem or task. Test for expert system suitability, 

2. Select a tool to build the expert system. 

3. Acquire and analyze knowledge of the domain-specific area. 

4. Construct several example cases that the expert system will solve. 

5. Build a prototype. 

6. Test, revise, and expand the prototype. 

A. STEP 1: IDENTIFY AND DEFINE A TASK 

The task of troubleshooting helicopter system problems was chosen for for this 
study for several reasons. The author's experience as a helicopter squadron Post 
Maintenance Functional Check Pilot has indicated that "good" troubleshooting skills 
are particularly difficult to obtain. These skills are difficult to obtain because of the 
extensive training and experience required. The lack of comprehensive documentation 
of helicopter problems and troubleshooting procedures intensifies this difficulty. 

Once identified, the task area was tested for expert system feasibility using the 
criteria outlined by Hayes-Roth, Waterman, and Lenat [Ref. 5: p. 160], and the 
Teknowledge "M.l Sample Knowledge Systems" manual [Ref. 9: pp. 11-5 - 11-8]: 

1. The task area should ", . .focus on a narrow specialty that does not involve a lot of 
common sense knowledge" [Ref. 5: p. 160]. Most of what is known about H-46 
diagnostic procedures is formal knowledge contained in the maintenance 
manuals. It is well defined, specific, and accepted by the experts. For these 
reasons this task meets the domain-specific criteria, and does not require the 
exclusive use of common sense knowledge for problem-solving. 

2. Good domains and tasks are knowledge-intensive. Knowledge-intensive tasks are 
domain areas in which higher competence levels are gained through 
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increased knowledge of and experience in the task. Because of this relationship 
between competence and knowledge, knowledge-intensive tasks are 
characterized by considerable differences in peoples' performance ability. The 
knowledge domain of troubleshooting helicopter problems fits this definition of 
a knowledge-intensive task. 

Facts are accessible and generally understood by typical users. Facts and 
terminology about troubleshooting H-46 helicopters are familiar to maintenance 
people. Novice troubleshooters generally know how to hook up and use 
equipment such as voltage meters to gather facts about a problem. The 
diagnostic expert system could ask the troubleshooters for voltage meter test 
results, which they would understand. 

4. Facts are stable for the duration of a consultation. Facts about a helicopter 
problem generally remain true or false throughout a troubleshooting session. 
For example, if a troubleshooter is using a voltage meter to gather information 
about a failed generator, the voltage readings will remain the same during a 
particular troubleshooting procedure. The problem does not change. 

5. Solutions are enumberable. If ail possible solutions to a problem can be listed in 
advance the solutions are enumerable. The maintenance manuals' 
troubleshooting tables list most of the possible causes for all system symptoms. 
Those not covered in the tables can be acquired from the experts. 

6. Entities in the domain are discrete. Flelicopter system problems can be readily 
described as facts, values, and objects. For example, the isolation procedures 
for troubleshooting the symptom "No.l GEN FAIL light On" is to test for 
generator terminal voltages, and cross-connect supervisory panels, checking if 
the generator comes on line. These conditions are discrete entities with specific 
values. The values can be "true" or "false", "yes" or "no", or numeric. They are 
not continuous or "fuzzy" like trying to describe the "attractiveness" of an 
financial investment. 

7. The task passes the "telephone test". The domain area of a troubleshooting 
problem passes the telephone test as described in the Teknowledge "M.l Sample 
Knowledge Systems" manual: 

A competent performer could help the intended user of your knowledge system 
over the phone. This means that a verbal dialog is sufficiently rich to capture 
the important facets of the problem. If perception plays an important role in 
problem solving, then the task will fail the telephone test. The knowledge of a 
master mechanic who depends on the sounds and odors of the engine will be 
difficult to capture, for example, unless the mechanic and end-user are able to 
communicate about these cues unambiguously. [Ref. 9: p. 11-8] 

After identifying a problem and testing for expert system feasibility, the scope 
and limitations of the knowledge base were defined. These are described in Chapter I. 
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B. STEP 2: SELECTING A TOOL 



The knowledge system development tool marketed by Teknowledge, Inc., known 
as M.l, was selected for use on this project after consideration of the available tools. 
The M.l tool has many advantages. First, M.l is designed to prototype 
diagnosis/prescription consultations [Ref. 4: p. 179]. The problem selected for this 
study concerns such consultations. 

Second, programming with M.l is relatively easy to learn, and resulting programs 
are easy to use. It is programmed using "if-then" rules written in near plain English. 

Third, M.l has been applied to automotive diagnostic systems. Diagnostic 
processes are basically similar, which makes M.l an excellent tool for the problem 
considered in this study. 

Fourth, M.l is available for educational purposes at the Naval Postgraduate 
School by agreement with Teknowledge. The documentation and teaching materials 
that accompany the M.l program diskettes are clear and easy to read. 

Last, the knowledge base program can be written using any IBM PC compatible 
word processor. Both SIDEKICK by Borland International, Inc., and WORDSTAR 
by MicroPro International Corp., which have similar editing commands, were used. 
SIDEKICK has the advantage of allowing the author to alternate rapidly between the 
running program and the text editor to make changes to the knowledge base. This 
facilitates debugging and adding rules. However, SIDEKICK has a limit of 
approximately 1300 lines of text or program code per notepad file. When a knowledge 
base module exceeded this limit, WORDSTAR was utilized. 

C. STEP 3: ACQUIRE AND ANALYZE KNOWLEDGE 

Knowledge acquisition (KA) is probably the most difficult step in building expert 
systems. It involves ". . .the transfer and transformation of problem-solving expertise 
from some knowledge source to a program" [Ref. 5: p. 129]. Mark D. Grover describes 
KA as a three phase process: domain definition, fundamental knowledge formulation, 
and basal knowledge consolidation. The three phases are shown in Figure 4.1 
[Ref. 10]. 

KA is pertinent throughout expert system development. Because of this 
importance, Grover's KA cycle was integrated with the six steps of the CADS 
development strategy. The relationship between the KA cycle and the six development 
steps is shown in Figure 4.2. 
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Figure 4.1 Knowledge Acquisition Cycle. 

The first phase of the KA cycle is described by Grover as: 

... the careful understanding and recording of the domain. The goal is the 

production of a Domain Definition Handbook, containing: 

• General problem description, 

• Bibliography of reference documents, 

• Glossary of terms, acronyms, and symbols, 

• Identification of authoritative "experts," 

• Definition of appropriate and realistic performance metrics, and 

• Description of example reasoning scenarios. [Ref. 10] 

This research documentation is considered the Domain Definition Handbook for 
this project. The general problem description and the identification of reference 
documentation were accomplished in step one of the development process. The 
knowledge engineer then must become familiar with the knowledge domain. This is 
done by acquiring knowledge from books, manuals, technical publications, and 
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Figure 4.2 KA Cycle and CADS Development Relationship. 

pamphlets. The objective is to investigate and become familiar with any structured 
problem-solving methodology that the domain utilizes. The author as the knowledge 
engineer familiarized himself with the H-46 diagnostic process. The knowledge and 
process were analyzed to prepare for the construction of sample cases, which are 
discussed in step four. The bibliography of reference documents for CADS consists of 
the H-46 maintenance manuals. 

A glossary was initiated (contained in Appendix A) to include acronyms and 
expressions which are characteristic of Naval aviation maintenance. 

Identification of experts in the domain was accomplished by asking the Naval Air 
Systems Command H-46 Program Manager (N’AVAIRSYSCOM PMA-261) to identify 
technical representatives who are knowledgeable about the recently modified H-46. 
Five Naval Aeronautical Engineering Service Unit (NAESU) technical representatives 
were identified. They are located at Marine Corps Air Station Tustin, California, and 
Naval Air Station North Island, California. 
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A discussion of the performance metrics and formulation of sample cases is in the 
next section. 

D. STEP 4: CONSTRUCT SEVERAL SAMPLE CASES 

Phase two of Grover's KA cycle is fundamental knowledge formulation, which 
includes steps three, four, and five of the CADS development process. In this phase 
initial cases are selected and formulated for solution by the expert system. 
Fundamental knowledge should represent situations that are typical, well understood, 
and have expected conclusions. When the initial cases have been formulated, the 
identified domain expert reviews the cases for correctness. "This review forms a 
baseline for minimum performance, predictable testing and correction, and careful 
delineation of capabilities which can be expanded and subjected to experimentation" 
[Ref. 10]. This core of reviewed initial cases becomes what Grover calls the 
Fundamental Knowledge Corpus. 

Constructing the initial sample cases can be considered the first step toward 
designing the system and identifying the performance criteria. The performance metrics 
of the system were defined as follows: 

1. The goal of the initial cases, and ultimately the entire system, should be 
identified. W r hat is the system supposed to solve? 

2. The isolation procedures should be displayed to the user. 

3. The system should ask the user questions about the outcomes of the performed 
isolation procedures. 

4. The user should be given a list of possible answers to the questions that the 
system asks. 

5. Based on the user's responses the system should display the correct conclusion 
about the helicopter problem (goal). 

WTiat is the troubleshooter's goal? The goal is to find the cause of a helicopter 
system problem. In terms of M.l: 

goal = problem-cause. 

Finding the actual cause of a helicopter problem is paramount. Once the actual cause 
is known then the correct appropriate action can be taken, instead of merely changing 
components, without really identifying whether the component contains the fault. 

After the goal of the system's knowledge base was identified, the initial scenarios 
were constructed from the symptoms and isolation procedures of the maintenance 
manuals' troubleshooting tables. The problem causes listed in the troubleshooting 
tables were examined, then the isolation procedures were analysed to identify the 



39 



conditions that would lead a troubleshooter to those causes (goal). The outcomes and 
conditions of the isolation procedures became the antecedents of rules for which the 
problem-cause was the conclusion. Several rules were then drafted on paper, written in 
pseudo-English, i.e., the "if-then" format acceptable to M.l. 

Refering to the example discrepancy discussed in Chapter II, the No.l electrical 
generator malfunctions on the helicopter. One of the symptoms of this problem is that 
the No.l generator failure light has illuminated on the cockpit caution light panel. 
This symptom is listed in the troubleshooting tables as "GEN FAIL light On". See 
Figure 2.2. According to the troubleshooting tables, there are two outcomes of the 
isolation procedure (cross-connecting the supervisory panels): 1) the generator comes 
on line and the No.l GEN FAIL light extinguishes, or 2) the generator does not come 
on line and the light remains illuminated. If the generator comes on line then the 
supervisory panel should be replaced, implying that the cause is a faulty supervisory 
panel. The symptom and each of these outcomes are conditions which would lead a 
troubleshooter to this problem-cause. The rule can be written thus: 
if symptom = 'GEN FAIL light On' 
and 'generator comes on line' 
then problem-cause = 'Faulty Supervisory Panel'. 

The CADS program prompts the user for the symptom and possible values for 
the conditions. In M.l each of these conditions is recognized as an expression. M.l 
then uses meta-facts to determine the values of the expressions. Meta-facts are 
knowledge base entries that provide information or directions about how to determine 
a value. . . [Ref. 9: p. 4-6]. The most common meta-fact used is a question: 
question('generator comes on line') = 

['Having performed the isolation procedure, 
does the generator come on line?']. 

The answers to this question are displayed using the M.l meta-fact legalvals, which list 
the legal values of the expression 'generator comes on line'. The possible responses to 
this question are written in the meta-fact: 

legalvals('generator comes on line') = ['yes', 'no']. 

Had the user answered 'yes', the expression 'generator comes on line' would be true, 
and the problem-cause displayed, ending the consultation. 

Rules and meta-facts were written on paper following this logic for several 
problem-causes. A set of meta-facts and rules leading to a conclusion (goal) are 
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considered a case or scenario. These meta-facts and rules were then coded into the 
program for an initial prototype, which is discussed next. 

E. STEP 5: BUILDING A PROTOTYPE 

rhe strategy for building the initial CADS prototype w r as to use the adaptive 
design method, better known as prototyping. Using the prototyping method, a 
knowledge engineer can 1) build a working system quickly, 2) evaluate the 
appropriateness of the initial design quickly without investing large amounts of time 
and energy into an unworkable design, and 3) make adaptions to the design, user 
interfaces, and outputs. Once an initial working prototype is developed, the knowledge 
engineer can modify and refine the program as necessary 7 . 

The ability to make changes and adaptions to the CADS knowledge base system 
design is important. When the initial CADS prototype was completed, interviews with 
the H-46 maintenance experts were conducted to acquire their knowledge of 
troubleshooting techniques and procedures. They reviewed the MI Ms troubleshooting 
tables for correctness and added new information. The prototype CADS knowledge 
base was modified accordingly, to include this expert knowledge. 

Using the prototyping approach, the first set of rules, questions, and meta-facts 
were written, entered into M.l, and run. Logic flow and solutions to helicopter 
discrepancies w r ere compared with those given in the maintenance manuals. As CADS 
yielded the correct problem-cause, more rules and knowledge base entries were written. 
The knowledge bases for each sub-system of the electrical and hydraulic systems were 
completed in this iterative manner. 

The logical and physical structures of the CADS knowledge base are similar. 
The knowledge base is divided into modules. Each module represents a subsystem 
knowledge base. The electrical system is divided into three subsystems: AC system, 
DC system, and APU system. The hydraulic subsystems are the flight control system, 
flight control pressure indicating system, utility power system, and the utility pressure 
indicating system. This structure is shown in Figure 4.3. 

Using this "divide and conquer" approach, each knowledge base module was 
written independently. This structure was chosen over the option of having one, 
immense knowledge base. Modularizing the CADS knowledge base allowed the 
knowledge engineer to concentrate on a specific subsystem, enhancing the ability to 
debug and revise. 
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Figure 4.3 CADS Knowledge Base Structure Chart. 

Modularizing requires that there be a top-level control module to initialize the 
CADS program, and to load the appropriate knowledge base file. The top-level 
module begins the CADS consultation by offering the user directions and asking in 
which major system and subsystem the aircraft problem is located. The appropriate 
knowledge base is then loaded. 

F. STEP 6: TEST, REVISE, AND EXPAND THE PROTOTYPE 

Grover's third phase of the KA cycle is basal knowledge consolidation, which 
relates to steps three, five, and six of the CADS development strategy. This phase is 
an iterative process with the objective of testing, improving, and expanding what 
Grover calls the Fundamental Knowledge Corpus. Basal knowledge can be considered 
a core knowledge base system that meets the minimum performance metrics listed in 
step four of the system-building process. 
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1. Testing and Verification 

Testing the CADS prototype involved evaluating the advice that was displayed 
(isolation procedures), the data acquisition process (questioning), and the conclusions 
reached. 

In defining the test criteria the following questions were considered: 

• Does the top level module initialize the CADS program properly? 

• Is the correct knowledge base loaded? 

• Are the correct isolation procedures provided in the correct sequence? 

• Are the right questions being asked in the correct sequence, and for the right 
reasons? 

• Are the conclusions correct? Given the responses to the questions, is CADS 
finding and displaying the correct problem-cause for that combination of 
conditions? 

• Are the isolation procedures, conditions, and conclusions consistent with the 
experts? 

To answer these questions, testing was performed in two phases. The first 
phase of testing involved evaluating the system for consistency with the maintenance 
manuals' troubleshooting tables. Because of the known inadequacies of the 
maintenance manuals (out of date, inaccurate, lack of information), a second phase of 
testing and verification was necessary'. This second phase was to check the prototype 
knowledge base for consistency with the experts: the five NAESU technical 

representatives. 

The first phase of testing was conducted throughout prototype development. 
As the rules were written, each conclusion was manually checked to determine that the 
antecedents (conditions) leading to it were correct. Each antecedent must have a value 
either solved for by the system, provided by the user, or stored in the cache, for the 
correct conclusion to be reached. This ensures that questions prompting the user for 
values are asked at the right time and under the right conditions. CADS has been 
tested and works according to the information contained in the troubleshooting tables. 

As mentioned above in step four, problem causes listed in the troubleshooting 
tables were examined, and the isolation procedures were analyzed to identify the set of 
conditions that would lead to those causes. Many inconsistencies were noted in the 
troubleshooting tables. The sequence of isolation procedures was sometimes vague. 
Outcomes to procedures were missing or suspect. A copy of the electrical and 
hydraulic systems troubleshooting tables, with these notations, was sent to the experts 
for clarification, initiating phase two testing. 



43 



The objectives of phase two testing were a review of the CADS knowledge 
base by the experts, and acquisition of the heuristics of the experts. The experts had 
approximately three weeks to review and comment on the noted discrepancies of the 
MI Ms tables. Then face-to-face interviews with the experts were conducted. 

During the interviews, the experts and knowledge engineer reviewed the 
discrepancies noted in the troubleshooting tables. The experts provided up-to-date, 
correct information of the electrical and hydraulic systems troubleshooting procedures. 
They also had the opportunity to run and evaluate the CADS prototype. 

2. Revision 

Based on the experts' review of the CADS prototype and the troubleshooting 
tables, revision of the knowledge base took place. Rules, display of isolation 
procedures, and questions were rewritten as necessary. After this revision, CADS was 
considered a completed prototype for the purposes of this project. Its knowledge base 
contains both formal and heuristic knowledge of the experts. 

As discussed above, basal knowledge consolidation is an iterative process. An 
expert system's knowledge base is continually reviewed, up-dated, and improved by the 
knowledge engineer in collaboration with the experts. However, further iterations of 
CADS are beyond the scope of this project. 

3. Expansion 

The rules and meta-facts were revised after the experts' review. There are no 
plans to expand the CADS knowledge base further at this time. 
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V. RESULTS, CONCLUSIONS, AND RECOMMENDATIONS 



A. RESULTS 

Improper troubleshooting techniques result in lost time, unnecessary removal of 
system components, and in wasted resources. These problems are not exclusive to the 
Navy's H-46 helicopter community, but are common throughout the military. A 
possible solution to these problems is the application of expert systems technology -- 
not only to the Navy's H-46 helicopter community, but to other military maintenance 
processes. 

This study was undertaken to demonstrate the feasibility of applying expert 
system technology to the Navy's H-46 helicopter maintenance process. A 
microcomputer-based prototype diagnostic system known as CADS was developed for 
this purpose. The prototype CADS does what it was designed to do: aid H-46 
squadron maintenance personnel find the cause of helicopter electrical and hydraulic 
systems problems. Therefore, the answer to the main question addressed in this study 
is "yes", a computer-aided diagnostic system is feasible and and applicable to the 
Navy's H-46 helicopter maintenance process. The success of CADS suggests that 
expert system technology may be appropriate for general application to military 
maintenance systems. 

B. CONCLUSIONS ABOUT M.l AND CADS 

1. Application of M.l to structured selection problem-solving 

The M.l knowledge system development tool is particularly suitable for 
building knowledge bases that use a structured selection approach to problem-solving. 
Structured selection is a problem-solving methodology by which the problem-solver 
systematically progresses through a richly structured search space of enumerated 
problems. These enumerated problems have been seen and solved before. The task is to 
select and refine a solution for a particular case rather than create it anew. 

It has been shown in this research that diagnosing helicopter problems uses 
such a structured selection approach to problem-solving. Symptoms and causes of 
helicopter problems are enumerated and contained in the maintenance manuals 
troubleshooting tables. Most or all of the problems have been solved throughout the 
history of the H-46 helicopter. Troubleshooting of these problems, plus any new ones 
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that arise, is conducted daily. As a result, diagnosing helicopter problems provides a 
good example of an M.l problem-solving application. 

2. Memory requirements for CADS 

The prototype CADS developed for this research is contained on two 
5.25-inch, 360K byte floppy diskettes. One diskette contains the CADS initialization 
module and the CADS executable file, using 246K bytes of disk space. The second 
diskette contains the modularized subsystem knowledge bases, which use 139K bytes of 
disk space. These knowledge base files can be called by the initialization module 
whether they remain on the diskette or have been copied to a hard disk drive. 

The CADS executable file can be configured for use on two floppy disk drives 
or on a hard disk drive. Use of floppy diskettes in the two drive configuration ensures 
portability of the program. However, the recommended configuration is for use on a 
hard disk drive, for several reasons. First, a CADS knowledge base is loaded much 
faster, and the inference process executes much faster on a hard disk drive: knowledge 
base files can be called and loaded within 30 seconds by the initialization module. This 
operation can take over five minutes in the two drive configuration, depending on the 
knowledge base size (number of rules and meta-facts). 

Second, the hard drive configuration is recommended in view of the recent 
contract for Navy-wide implementation of Zenith 248 microcomputers. Currently, 
each squadron has one such microcomputer. It is anticipated that more will be 
installed in the future. 

3. Modularizing the CADS knowledge base 

Modularizing the CADS knowledge bases has several advantages. First, a 
more efficient use of memory space is realized by having only the initialization module 
and the executable file (both on CADS diskette #1) remain resident in RAM. Much 
less microcomputer memory space is required with this configuration. The knowledge 
base files remain on CADS diskette # 2, and are accessed as necessary. 

Second, the inference process is more efficient. Since only one knowledge base 
module is loaded at a time, the inferencing process searches for values and rules 
pertinent to that particular helicopter system. Computer processing time and memory 
is not wasted in searches down branches of irrelevant systems. 

Third, modularity allows for expansion of the system, without having to 
increase the requirement for the microcomputer's RAM space. Because information 
about the helicopter systems is divided into separate knowledge base files, additional 
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modular knowledge bases could be developed and written, independent of the original 
prototype. These additional knowledge bases could be integrated into the system, with 
little modification required of the initialization module. The modification basically 
requires the addition of valid answers to existing questions, and the addition of a rule 
to load each new knowledge base. 

4. Using the trace function to demonstrate reasoning 

CADS provides an excellent training tool for inexperienced troubleshooters. 
However, the original proposal for using the trace function to show the reasoning 
process has not been satisfactory’ in this M.l implementation. 

M.l's inference tracer displays information describing the inference process. 
Also displayed are questions, menus, prompts, expressions being sought, which rules 
have succeeded or failed, and which knowledge base entries are being fired. The trace 
function is intended for use by the knowledge engineer in the development stage, for 
finding subtle faults in the knowledge base. As a result, M.l's trace function traces 
through the knowledge base seeking values of rules which are not relevant to finding 
the cause of the problem. This reasoning is not necessarily similar to an expert's 
reasoning method. 

Another disadvantage of the trace function is that rules and expressions are 
displayed exactly as they are written in the knowledge base. Readability of the rules 
and expressions displayed becomes a function of the programmer's writing style. 
Although the rules are written in "if-then" format using near plain English, they may 
seem cryptic to a user unfamiliar with the language of expert systems and M.l. 

5. Training knowledge engineers for the Navy 

The possibility exists for new system discrepancies to appear throughout the 
H-46 fleet. If CADS were implemented, there would be a need for people within the 
helicopter squadrons who are able to document and program this new information into 
existing CADS knowledge bases. These people must be familiar with the language of 
expert systems, and in particular with the use of the M.l knowledge development tool. 

Because of the ease of developing expert systems utilizing this tool, people 
with little or no programming experience could be trained for this purpose. The U.S. 
Army Signal School at Ft. Gordon, Georgia, has such a training program. Army 
personnel with expertise in particular technical areas, but with no programming 
experience, attend a two-week training course. They learn how to use M.l, and 
develop small useable systems within that time. The Navy could set up a similar 
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school for this purpose. People with this valuable training could return to their 
commands where they could maintain existing CADS software, plus apply this 
technology to other problem-solving areas, developing new and useful systems. 

Knowledge gained from this type of training would be particularly important 
for improving and expanding the "corporate knowledge" of troubleshooting expertise. 
As new problems arise and are diagnosed, and as existing procedures are improved and 
updated, this new knowledge can be programmed and put to immediate use. This 
could prove to be an advantage over printing changes on paper, for incorporation into 
updated printed manuals. 

C. SUGGESTIONS FOR FURTHER STUDY 

1 . Knowledge base expansion 

The next step after prototype development is to expand the prototype CADS 
into a fully developed expert system containing all the diagnostic information of all the 
helicopter systems. A fully developed CADS could also contain the corrective action 
procedures for each helicopter system discrepancy, and other amplifying maintenance 
information. Such a long term project might involve a more structured analysis and 
design of diagnostic information about the entire helicopter. However, the prototyping 
development approach could be integrated with the information structuring. 

Additional knowledge acquisition would also be required to extract all 
pertinent knowledge from the maintenance manuals and the experts. This would 
involve more interviewing and survey hours than that required for the development of 
the CADS prototype. 

If CADS were expanded to include other helicopter systems, more secondary 
memory would be required for the program. Including all helicopter subsystems as 
additional knowledge bases would increase the requirement for floppy diskettes 
significantly. If the user has a microcomputer with only two floppy disk drives, it will 
be necessary to load and unload the diskettes, whenever troubleshooting different 
helicopter systems is required. Also, loading and inferencing times will be significantly 
slower. Although the expanded CADS would be portable, it is recommended that an 
expanded system be used on a computer with a hard disk drive. 

2. Enhanced use of M.l explanation facility 

The prototype CADS could be improved by providing more specific 
explanations to users, enhancing readability and making the system more expert. This 
could be accomplished by utilizing the M.l explanation facility. Customized 
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explanations, amplifying information about a path of reasoning, or elaboration of 
heuristic knowledge would be displayed when users enter why in response to a 
question. Ordinallv, if a user entered why, M.l would only display the knowledge base 
entry under consideration. 

3. Enhanced use of the "unknown" response 

The term unknown is always an acceptable answer to questions posed in M.l. 
This gives the M.l {and CADS) inference process the advantage of continuing to 
search for values even though a value has not been concluded. It causes the 
inferencing process to change its search to other branches of the knowledge base, 
allowing for the solution of other values, to seek a conclusion. The prototype CADS 
knowledge base could be expanded to provide additional information should the user 
select unknown as an answer. This is accomplished utilizing the M.l control command 
whenfound to display a text string of additional information. 

4. Integration with graphics 

Ultimately, a fully developed CADS could integrate all diagnostic information, 
figures, diagrams, and maintenance procedures. Through the use of "pull down" 
windows, all information would be available on the computer for the maintenance 
person. This could eliminate the need for printed manuals. 

Integration with graphics software could be an initial step toward such a fully 
developed system. M.l has the ability to access graphics programs through C language 
patches. The use of graphics to describe system components would greatly enhance 
the ability of maintenance personnel to troubleshoot problems. With graphics, 
component location and troubleshooting procedures could guide personnel through 
maintenance procedures. This could be extremely useful if the maintenance personnel 
are unfamiliar with maintenance procedures. 

Graphics enhancements such as color, rotating three-dimensional views of 
components, and rotating transparent views of the helicopter are also possibilities for 
future study. The use of these graphics techniques would facilitate troubleshooting. 
Used in a training environment, such graphics techniques could make learning 
helicopter troubleshooting an exciting, motivating experience, especially for a 
generation of young mechanics raised with video games. 

5. Integration with CD-ROM technology 

Other futuristic possibilities exist for a CADS integration with Compact Disc 
Read Only Memory (CD-ROM) technology. This not only would provide an 
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extremely portable medium, but would significantly reduce or even eliminate literally 
tons of printed paper. This consideration is of particular importance in relation to 
helicopters deployed aboard ship. Eliminating the need for shipboard office space 
containing tons of technical manuals is critical. 

6. Study of CADS implementation in the squadrons 

Implementation of CADS in the squadrons and in deployed units offers many 
interesting possibilities for further study. This research could be conducted using the 
prototype or a fully developed system. However, a study using a fully developed 
system would be more desirable, to evaluate the success of implementation. Prior to 
intiation of any studies, an implementation plan should be developed. Studies could 
include the amount of training required to familiarize maintenance personnel in the use 
of microcomputers and CADS. Comparisons could be made of the learning curves for 
computer literate and illiterate users. 

D. RECOMMENDATIONS 

The following is a list of recommendations concerning the use of CADS and M.l 
in general: 

1. Use the M.l knowledge-based system development tool for building knowledge 
bases that use a structured selection approach to problem-solving. 

2. Implement CADS on a hard disk drive system for efficient use of memory. 

3. Keep the knowledge bases modular in large system applications for efficient use 
of memory space, efficient inference processing, and ease of system expansion. 

4. Train Navy personnel to use M.l to develop immediate and useful systems, and 
to maintain implemented systems. 

5. Expand the CADS knowledge base to include the rest of the helicopter's 
systems. 

6. Improve the M.l explanation facility, and the "unknown" response to provide 
more specific information to users, enhance readability of the system, and make 
the system more expert. 

7. Add graphics capabilities as a first step toward a fully developed system. 
Helicopter component descriptions utilizing graphics would enhance 
troubleshooting. 

8. Investigate the possibilities of integration with CD-ROM technology. 

9. Develop a plan for implementation of the system in the helicopter squadrons. 
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VI. SUMMARY 



A microcomputer-based prototype diagnostic system, known as CADS, has been 
developed to demonstrate the feasibility of applying expert system technology to 
helicopter maintenance. This system diagnoses electrical and hydraulic system 
problems in the Navy's CH-46D S,R,&M helicopter. 

As an interactive program, CADS prompts the user for information with a 
sequence of questions, and provides valid answers. The sequence of questions depends 
upon the answers that the user selects. As the questions become more specific, the 
program searches the knowledge base for valid causes of the helicopter system 
problem. Searching is accomplished utilizing a backward chaining, decision tree 
technique to find solutions. 

Chapter II discusses the Naval aviation maintenance process, current 
troubleshooting procedures, and how CADS could be integrated into the 
troubleshooting process. The written references for the CADS knowledge domain are 
two volumes of 15 maintenance information manuals. These two volumes contain the 
electrical and hydraulic systems. The prototype CADS solves over 39 specific 
symptoms referred to in over 28 pages of troubleshooting tables. Although this 
represents only a small portion of the entire published helicopter diagnostic 
information, it provides enough data for a realistic assessment of CADS. The 
secondary research questions listed in Chapter I, pertaining to squadron applications, 
are also discussed. 

Chapter III contains a description of knowledge-based expert systems, and the 
M.I knowledge-based system development tool. A comparison of knowledge-based 
systems and conventional programs is made. The criteria necessary for expert systems 
to be useful and to perform at a significant level of expertise (as suggested by Barr and 
Feigenbaum) is provided. The description of M.I includes its capacity of rules and 
facts, how the inferencing process works, and how M.I handles uncertainty. 

Chapter IV discusses in detail the development of the prototype CADS, and how 
it works. The strategy used to develop CADS is a combination of three methodologies, 
resulting in six development steps. Because of the important role that knowledge 
acquisition has throughout expert systems development, the knowledge acquisition 
cycle was integrated with the six steps. This relationship is discussed in detail. 
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Logically, the CADS knowledge base is divided into modules. Each module 
consists of the knowledge base for a helicopter subsystem, such as the AC electrical 
subsystem. The advantages of modularizing the knowledge base are discussed. Criteria 
for a two phase test plan were developed. After testing the prototype CADS, revision 
of the knowledge base modules was accomplished. 

A glossary of acronyms used throughout this research is contained in Appendix 
A. Appendix B contains a sample CADS consultation using an example electrical 
system problem. The CADS knowledge base (source code) is in Appendix C. 

Based on this discussion, it has been shown that expert systems are well suited 
for diagnostic applications. The technology, hardware and software exist and are 
available for use in such an expert system. 
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APPENDIX A 

GLOSSARY OF ACRONYMS 



AC 

AI 

APU 

CADS 

CD-ROM 

CF 

CH-46D 

CNO 

DC 

FY 

GEN FAIL 

HC 

H-46 

JCN 

KA 

kb 

MAF 

MDR 

MDS 

MIM 

MIS 

MMH/FH 

NAESU 

NAMP 

NAVAIRSYSCOM 

PMA-261 

RAM 



Alternating Current. 

Artificial Intelligence. 

Auxiliary Power Unit. 

Computer-Aided Diagnostic System. 

Compact Disc Read Only Memory. 

Certainty Factor, the degree of confidence one has in a fact or 
relationship. In M.l, CF is assigned a value between + 100 and 
- 100 . 

Designation of S,R,&M Sea Knight helicopter. 

Chief of Naval Operations. 

Direct Current. 

Fiscal Year. 

Generator failure. 

Designation for Helicopter Combat Support Squadron. 

Designation for the Sea Knight helicopter used in the United 
States Navy and Marine Corps for utility missions. 

Job Control Number. 

Knowledge Acquisition. 

Knowledge base entry. 

Maintenance Action Form, for documentation of aircraft 
discrepancies. 

Maintenance Data Report. 

Maintenance Data System. 

Maintenance Information Manual. 

Management Information System. 

Maintenance Man-hours per Flight Hour. 

Naval Aeronautical Engineering Service Unit. 

Naval Aviation Maintenance Program. 

Naval Air Systems Command. 

NAVAIRSYSCOM Program Manger for H-46 squadrons. 
Random-access memory: computer's high speed (main) memory. 
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S,R,&M 



TD 

VIDS/MAF 



Surviveability, Reliability, and Maintainability Program, that 
extends the service life of the H-46 helicopter through the year 
2000. 

Technical Directive. 

Visual Information Display System/ Maintenance Action Form. 
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APPENDIX B 

INTRODUCTORY USER'S GUIDE TO CADS 



1. INTRODUCTION 

CADS is a prototype microcomputer-based expert system designed to help 
squadron and detachment maintenance people troubleshoot S,R,&M CH-46D 
helicopter system problems. The goal of the CADS prototype is to find the exact 
cause of electrical and hydraulic system discrepancies ("gripes"). CADS contains 
limited corrective action procedures. 

The purpose of this Introductory User's Guide is to get a troubleshooter started 
and somewhat familiar with use of the CADS prototype. It is not meant to be a 
comprehensive instruction manual explaining all the various capabilities and 
peculiarities of the CADS software. 

The following assumptions apply to using the CADS prototype: 

a. Users are familiar with the use of a microcomputer. 

b. Users have access to a Zenith 248 or IBM PC compatible microcomputer, with 
two disk drives or a hard drive. 

c. Users have a copy of the CADS prototype program, either on the two 5.25 inch 
floppy diskettes, or on the hard drive of the microcomputer. 

2. WHAT TO EXPECT 

CADS helps troubleshooters find the cause of a helicopter discrepancy by 
displaying isolation procedures for the troubleshooters to perform, then asking 
questions about the outcomes of the performed procedures. CADS lists all the 
possible answers to questions it asks. You as the troubleshooter select the appropriate 
answer based on the outcomes of the isolation procedures you have performed. 

CADS is a computerized version of the maintenance manuals' troubleshooting 
tables. The intention is to demonstrate that maintenance information can be made 
accessible on disks and microcomputers, so you do not have to deal with the 
maintenance manuals. 

The best way to approach CADS is to imagine that you are consulting the 
troubleshooting tables. The difference is the added feature that CADS will ask you the 
right questions about the isolation procedures you have performed, at the right time, 
and for the right reason. You will not have to waste time searching the manuals for 



55 



